Day 5: self-healing, interruption, and model-API retry#5
Conversation
The loop detector fingerprints each step's (tool, args) and, when a step repeats the previous one exactly, appends a "change your approach" note to the tool observation so the model breaks out of a rut. interrupt() sets an asyncio.Event that is checked at the top of each step. When set, the run yields a "Run interrupted." FinalEvent and stops without recording a FinalStep, so run(reset=False) resumes from the steps already in memory. This is a graceful pause between steps, not a mid-await crash. Also tidies this file's comments to plain prose and drops the numbered process markers.
_create_with_retry now retries rate limits, connection and timeout errors, and 5xx responses with exponential backoff, then re-raises once max_retries is exhausted. That is the agent's one fatal path. Permanent errors such as a bad request or bad auth are not retried and propagate immediately. Also tidies the module and method docstrings to plain prose.
PR Summary by QodoAdd loop detection, interruption, and transient model-API retry guardrails
AI Description
Diagram
High-Level Assessment
Files changed (5)
|
Code Review by Qodo
1. Loop nudge overclaims result
|
| # Fingerprint this step's calls: (name, canonical JSON args) each. | ||
| signature = tuple( | ||
| (tc.name, json.dumps(tc.arguments, sort_keys=True)) | ||
| for tc in response.tool_calls | ||
| ) | ||
| looping = signature == previous_signature | ||
| previous_signature = signature | ||
|
|
||
| # Announce every call, then run them (concurrently or in order). | ||
| for tool_call in response.tool_calls: | ||
| yield ToolCallEvent(tool_call=tool_call) |
There was a problem hiding this comment.
1. Loop nudge overclaims result 🐞 Bug ≡ Correctness
Agent._run_stream() appends a note saying the repeated tool call “got the same result,” but the loop detector only compares the (tool name, args) signature and never checks whether the tool output actually matched the previous step. This can mislead the model (and anyone inspecting observations) when tools are non-deterministic or when repeating a call is intentional.
Agent Prompt
### Issue description
The loop detector sets `looping` based only on an exact repeat of tool-call `(name, args)`, but `_LOOP_NUDGE` claims the call also “got the same result.” This is stronger than what the code verifies and can provide incorrect guidance.
### Issue Context
- `looping` is computed from the tool-call signature only.
- The nudge text is appended unconditionally whenever `looping` is true.
### Fix Focus Areas
- src/agentling/agent.py[29-35]
- src/agentling/agent.py[129-151]
### Suggested fix
Either:
1) Change `_LOOP_NUDGE` wording to avoid asserting “same result” (e.g., “you already made this exact tool call; try a different approach”), **or**
2) Track the previous step’s tool results (e.g., store `(signature, results_content)`), and only append the “same result” phrasing when both signature and results content match.
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
| if self._interrupt.is_set(): | ||
| self._interrupt.clear() | ||
| yield FinalEvent(answer="Run interrupted.") | ||
| return |
There was a problem hiding this comment.
2. Interrupt bypasses forced-final 🐞 Bug ☼ Reliability
The interrupt flag is only checked inside the main step loop; if interruption is requested late (e.g., during the last iteration), the agent can still execute the post-loop “step limit reached” forced final-answer model.generate() call. This triggers an unwanted extra model request after the run was asked to stop.
Agent Prompt
### Issue description
`interrupt()` is intended to stop the run before the next step, but the implementation only checks `_interrupt` at the start of each `for _ in range(limit)` iteration. If an interrupt is set during the last loop iteration, control can fall through to the step-limit forced-final-answer path, which still calls `model.generate()`.
### Issue Context
After the main loop exits, the agent unconditionally executes the forced-final-answer branch. This branch should also honor the interrupt request.
### Fix Focus Areas
- src/agentling/agent.py[113-118]
- src/agentling/agent.py[176-186]
### Suggested fix
Before executing the post-loop forced-final-answer `model.generate()`, add the same interrupt guard used in the main loop:
- if `_interrupt.is_set()`: clear it, yield `FinalEvent(answer="Run interrupted.")`, and return.
This ensures interruption is honored consistently even when the run is about to perform the forced final model call.
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
Implements FOL-42: production guardrails on top of the agent loop.
"change your approach" note to the observation so the model breaks the loop.
(yields "Run interrupted.", records no FinalStep) and run(reset=False)
resumes from the steps already in memory.
connection and timeout, 5xx) with exponential backoff, aborting after N.
Permanent errors such as bad request or bad auth fail fast.
Follow-ups filed during the coverage audit: FOL-45 (correctness issues) and
FOL-46 (exhaustive test coverage).
Closes FOL-42.